Splitting the hardware intrinsic tests into more test groupings #61973
Conversation
dotnet-issue-labeler
bot
added
the
area-CodeGen-coreclr
|
Tagging subscribers to this area: @JulieLeeMSFT Issue DetailsThis should help mitigate #60154 by ensuring that each test group is "sufficiently small". It should also help with the overall test throughput. The split is roughly on "ISA" but could be split as more fine-grained if required.
|
|
CC. @jkotas, @echesakovMSFT |
|
We will need to figure out how to fix these tests into the larger test refactoring that @trylek and @jkoritzinsky are working on. @trylek and @jkoritzinsky Do you have thoughts on this? Having said that, I think this change is a fine short-term fix for the GC stress timeout problem. |
|
My current thinking is that, according to Jeremy's initial implementation, the "new" group projects will be "intentional" in the sense that the grouping won't be automatic, we'll use structure of the test tree and performance of the individual test groups to decide where to put those. The grouping projects will ultimately replace the Helix grouping project and will be used to construct directly runnable Helix work items (I'm working on this new logic right now). Thus, once the conversion has been completed, we should be able to delete both the legacy xUnit wrappers and the |
|
I was thinking that maybe we need two different partitionings for these tests - one for GCStress pipelines (where tests for each API method - e.g. The reasoning for that is there is no perfect partitioning that fits all test scenarios. The finer partitioning would make running the tests in innerloop pipelines much slower, while the coarser one would almost always fail in GCStress pipelines. |
The goal here is to just have this as a short-term workaround until the larger test refactoring that @trylek and @jkoritzinsky are working on. The larger refactoring should take care of the requirements around appropriately parallelizing and handling timeouts, etc. This PR however should expand things just enough to get CI to stop failing until that happens as the new failures are largely only due to the xplat tests added for Vector64/128/256 and them all running as part of the same group without this change. |
I would hope that we can avoid that by simply applying a test timeout multiplier for GCStress legs. |
There was a problem hiding this comment.
The goal here is to just have this as a short-term workaround
@tannergooding I should've been more specific that I was not suggesting to implement what I described in this PR. I was simply describing an option.
As a workaround the changes LGTM.
ghost
locked as resolved and limited conversation to collaborators
This should help mitigate #60154 by ensuring that each test group is "sufficiently small". It should also help with the overall test throughput.
The split is roughly on "ISA" but could be split as more fine-grained if required.